<html>
<head><meta charset="utf-8"><title>initial results · t-compiler/wg-pipelining · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/index.html">t-compiler/wg-pipelining</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/topic/initial.20results.html">initial results</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="164630892"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/195180-t-compiler/wg-pipelining/topic/initial%20results/near/164630892" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/topic/initial.20results.html#164630892">(May 01 2019 at 15:16)</a>:</h4>
<p><span class="user-mention" data-user-id="120989">@nnethercote</span> hm so I've started testing locally just to see some smoke test numbers with pipelining all enabled</p>



<a name="164630916"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/195180-t-compiler/wg-pipelining/topic/initial%20results/near/164630916" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/topic/initial.20results.html#164630916">(May 01 2019 at 15:17)</a>:</h4>
<p>If you download and build <a href="https://github.com/rust-lang/cargo/pull/6883" target="_blank" title="https://github.com/rust-lang/cargo/pull/6883">https://github.com/rust-lang/cargo/pull/6883</a> that'll give you Cargo (and you'll have to set <code>CARGO_BUILD_PIPELINING=true</code> as an env var)</p>



<a name="164630923"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/195180-t-compiler/wg-pipelining/topic/initial%20results/near/164630923" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/topic/initial.20results.html#164630923">(May 01 2019 at 15:17)</a>:</h4>
<p>and then I built a local rustc with your patch to emit metadata earlier</p>



<a name="164630931"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/195180-t-compiler/wg-pipelining/topic/initial%20results/near/164630931" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/topic/initial.20results.html#164630931">(May 01 2019 at 15:17)</a>:</h4>
<p>and unfortunately at least for Cargo itself it makes almost no difference for compiling from scratch</p>



<a name="164630990"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/195180-t-compiler/wg-pipelining/topic/initial%20results/near/164630990" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/topic/initial.20results.html#164630990">(May 01 2019 at 15:17)</a>:</h4>
<p>turns out Cargo's longest serial portion is a bunch of build scripts that probably shouldn't really be that serial</p>



<a name="164631002"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/195180-t-compiler/wg-pipelining/topic/initial%20results/near/164631002" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/topic/initial.20results.html#164631002">(May 01 2019 at 15:17)</a>:</h4>
<p>I saw a tiny bit of pipelining happening but it was nigh instantaneous</p>



<a name="164631076"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/195180-t-compiler/wg-pipelining/topic/initial%20results/near/164631076" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/topic/initial.20results.html#164631076">(May 01 2019 at 15:18)</a>:</h4>
<p>Cargo is probably a bad example though since it's just one big crate that depends on a huge number of little crates</p>



<a name="164631088"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/195180-t-compiler/wg-pipelining/topic/initial%20results/near/164631088" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/topic/initial.20results.html#164631088">(May 01 2019 at 15:18)</a>:</h4>
<p>I suspect pipelining is only really a win if there's two big crates</p>



<a name="164631176"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/195180-t-compiler/wg-pipelining/topic/initial%20results/near/164631176" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/topic/initial.20results.html#164631176">(May 01 2019 at 15:19)</a>:</h4>
<p>rustc also won't really get any benefit b/c it's all a bunch of dylibs which can't be pipelined</p>



<a name="164631291"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/195180-t-compiler/wg-pipelining/topic/initial%20results/near/164631291" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/topic/initial.20results.html#164631291">(May 01 2019 at 15:21)</a>:</h4>
<p>we should really get rid of those dylibs, ugh</p>



<a name="164631321"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/195180-t-compiler/wg-pipelining/topic/initial%20results/near/164631321" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/topic/initial.20results.html#164631321">(May 01 2019 at 15:21)</a>:</h4>
<p>It's currently being attempted in <a href="https://github.com/rust-lang/rust/pull/59800" target="_blank" title="https://github.com/rust-lang/rust/pull/59800">https://github.com/rust-lang/rust/pull/59800</a> which is a revival from earlier</p>



<a name="164631325"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/195180-t-compiler/wg-pipelining/topic/initial%20results/near/164631325" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/topic/initial.20results.html#164631325">(May 01 2019 at 15:21)</a>:</h4>
<p>but progress is slow from what it looks like</p>



<a name="164658771"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/195180-t-compiler/wg-pipelining/topic/initial%20results/near/164658771" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> njn <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/topic/initial.20results.html#164658771">(May 01 2019 at 21:18)</a>:</h4>
<p>@alexcrichton This definitely needs measurement across a range of workloads. The programs in rustc-perf might be a good place to start. (Note that you can't use the rustc-perf harness because that just measures compilation of the final crate.)</p>



<a name="164940200"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/195180-t-compiler/wg-pipelining/topic/initial%20results/near/164940200" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> njn <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/topic/initial.20results.html#164940200">(May 05 2019 at 22:48)</a>:</h4>
<p>@alexcrichton I'm happy to help with taking measurements, if you give me instructions on how to run Cargo with pipelining on</p>



<a name="165099888"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/195180-t-compiler/wg-pipelining/topic/initial%20results/near/165099888" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/topic/initial.20results.html#165099888">(May 07 2019 at 18:55)</a>:</h4>
<p><span class="user-mention" data-user-id="116015">@Alex Crichton</span> <span aria-label="point up" class="emoji emoji-1f446" role="img" title="point up">:point_up:</span></p>



<a name="165099994"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/195180-t-compiler/wg-pipelining/topic/initial%20results/near/165099994" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/topic/initial.20results.html#165099994">(May 07 2019 at 18:56)</a>:</h4>
<p>Sounds like we're primarily blocked on landing this in cargo nightly  or something?</p>



<a name="165119728"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/195180-t-compiler/wg-pipelining/topic/initial%20results/near/165119728" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> njn <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/topic/initial.20results.html#165119728">(May 07 2019 at 23:11)</a>:</h4>
<p><span class="user-mention" data-user-id="116009">@nikomatsakis</span> we need someone to make measurements. If it lands in Cargo nightly that will make it easier for people other than <span class="user-mention" data-user-id="116015">@Alex Crichton</span> to perform those measurements.</p>



<a name="165165216"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/195180-t-compiler/wg-pipelining/topic/initial%20results/near/165165216" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/topic/initial.20results.html#165165216">(May 08 2019 at 13:58)</a>:</h4>
<p>Ah yeah sorry I was just discussing my initial results, I still want to land pipelining in nightly Cargo to give others a chance to test.</p>



<a name="165166302"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/195180-t-compiler/wg-pipelining/topic/initial%20results/near/165166302" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/topic/initial.20results.html#165166302">(May 08 2019 at 14:12)</a>:</h4>
<p>OK. <span class="user-mention" data-user-id="116015">@Alex Crichton</span> do you think it's close to landing?</p>



<a name="165167003"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/195180-t-compiler/wg-pipelining/topic/initial%20results/near/165167003" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/topic/initial.20results.html#165167003">(May 08 2019 at 14:21)</a>:</h4>
<p><span class="user-mention" data-user-id="116009">@nikomatsakis</span> I believe so, yeah, although looks like ehuss found an issue which may take today to fix before it gets in</p>



<a name="165167009"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/195180-t-compiler/wg-pipelining/topic/initial%20results/near/165167009" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/topic/initial.20results.html#165167009">(May 08 2019 at 14:21)</a>:</h4>
<p>it'll take some time to get to rust-lang/rust as well due to the submodule usage</p>



<a name="165650860"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/195180-t-compiler/wg-pipelining/topic/initial%20results/near/165650860" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/topic/initial.20results.html#165650860">(May 14 2019 at 18:57)</a>:</h4>
<p>Looks like <a href="https://github.com/rust-lang/cargo/pull/6883" target="_blank" title="https://github.com/rust-lang/cargo/pull/6883">https://github.com/rust-lang/cargo/pull/6883</a> landed, at least -- but we still need a submodule update, <span class="user-mention" data-user-id="116015">@Alex Crichton</span> ?</p>



<a name="165651253"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/195180-t-compiler/wg-pipelining/topic/initial%20results/near/165651253" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/topic/initial.20results.html#165651253">(May 14 2019 at 19:01)</a>:</h4>
<p>roc apparently has some sort of degenerate case -- maybe we can get them to try it out :)</p>



<a name="165651849"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/195180-t-compiler/wg-pipelining/topic/initial%20results/near/165651849" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/topic/initial.20results.html#165651849">(May 14 2019 at 19:06)</a>:</h4>
<p><span class="user-mention" data-user-id="116009">@nikomatsakis</span> correct yeah, not on nightlies yet until submodule is updated</p>



<a name="165651859"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/195180-t-compiler/wg-pipelining/topic/initial%20results/near/165651859" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/topic/initial.20results.html#165651859">(May 14 2019 at 19:06)</a>:</h4>
<p>I plan to make a post on internals after it's available</p>



<a name="165652942"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/195180-t-compiler/wg-pipelining/topic/initial%20results/near/165652942" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Eric Huss <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/topic/initial.20results.html#165652942">(May 14 2019 at 19:18)</a>:</h4>
<p>fyi, I plan a submodule update soon.  There's a long queue of issues (the clippy update needs to land — which looks like it will miss tonight's nightly <span aria-label="sad" class="emoji emoji-2639" role="img" title="sad">:sad:</span> , then I need to land an rls PR, then do a combined rls/cargo update).  I really hope to make it before next week's beta, but the timing is getting tight.</p>



<a name="165684495"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/195180-t-compiler/wg-pipelining/topic/initial%20results/near/165684495" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> njn <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/topic/initial.20results.html#165684495">(May 15 2019 at 04:07)</a>:</h4>
<p><span class="user-mention" data-user-id="116015">@Alex Crichton</span> Have you managed to fix the issue you mentioned?</p>



<a name="165716443"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/195180-t-compiler/wg-pipelining/topic/initial%20results/near/165716443" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/195180-t-compiler/wg-pipelining/topic/initial.20results.html#165716443">(May 15 2019 at 13:57)</a>:</h4>
<p><span class="user-mention" data-user-id="120989">@nnethercote</span> I did yeah, just needs a submodule update now! (although I'm sure there's more bugs lurking)</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>